Due to such factors as advances in technology, reductions in computer hardware costs and growth of the world wide web, increasing quantities of digital data are being generated worldwide. For example, computer systems in homes, businesses and government are used to generate data in the form of text and other documents, databases, multi-media files, e-mail correspondence, web pages, and so forth. As a result, data storage demands are enormous and are growing over time.
An important aspect of data storage is in backing up the data for both data recovery and archival purposes. Backup solutions depend on some form of data redundancy and provide for the restoration of critical data that may be lost in the event of user errors, software errors, independent media failures (e.g., a failure of a disk drive or other component), correlated failures (e.g., a site disaster or power supply failure), and failed communication links. Data archival often requires that the data can be read back after an extended period of time.
Due to their cost effectiveness, magnetic tapes are most commonly used for backup storage. However, magnetic tapes have a number of drawbacks. For example, random accesses to data are difficult and time-consuming. This is because the tape may need to be mechanically spooled forward or backward in order to access certain data on the tape. In addition, tape systems typically have few tape drives relative to the number of tapes, resulting in low levels of parallelism. Also, tape drive head drift can result in a drive being unable to read data that was recorded by a different drive and may even prevent reading of data that was recorded by the same drive, but at an earlier time. Further, magnetic tape degrades over time, limiting its effectiveness for long-term storage and necessitating frequent testing.
Another important aspect of data storage is in obtaining faster, or nearline, access to data. For example, hierarchical storage management (HSM) systems provide varying accessibility to data by providing disk storage in conjunction with a tape or optical disk storage array or “jukebox.” Disadvantages of conventional HSM systems are similar those of tapes systems, as described above. For example, having to load media from the jukebox to a limited number of readers tends to make random accesses unacceptably slow.
It has been suggested that the prices of hard disk drives are approaching that of nearline tapes and, as a result, disks could replace tapes under certain circumstances. “Rules of Thumb in Data Engineering,” J. Gray and P. Shenoy, Proc. of Intl. Conf. on Data Engineering, February 2000. Disk drives, however, tend to have higher power requirements than other data storage devices, even when idle, and, thus, tend to be packaged less densely so they stay cool. In addition, disk drives tend to be inflexible in their handling of outstanding access requests in that typically only a small number of outstanding requests are permitted at any one time. Also, disk drives aren't used as removable media for archival purposes.
Therefore, what is needed are improved techniques for data storage. It is to this end that the present invention is directed.